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PREFACE 



This Manual should be of particular interest to potential users of the Leader 
Effectiveness Index {LEI), as well as to those who study leadership and its measurement. 
The Manual contains (1) the conceptualization of leadership which provides a foundation 
for the LEI (2) an explanation of how to use the instrument, (3) a description of the LEFs 
development and psychometric characteristics, and (4) a report of the process used to create 
an appropriate norm group. 

The LEI is a multirater instrument. It yields a single measure that reflects the 
effectiveness with which six tasks of leaders in vocational education are being performed. 
The six leadership tasks are to (1) inspire a shared vision and establish standards that help 
the organization achieve its next stage of development; (2) foster unity, collaboration, and 
ownership, and recognize individual and team contributions; (3) exercise power effectively 
and empower others to act; (4^ exert influence outside of the organization in order to set the 
right context for the organization; (5) establish an environment conducive to learning; and 
(6) satisfy the job-related needs of members of the organization as individuals. 

Individualized feedback to the person whose leadership performance is being rated 
(the ratee) shows the average rating assigned to the ratee by three to five of his or her 
subordinates and/or peers and a comparison of the average rating given the ratee with the 
average ratings given to a norm group of 551 vocational administrators and vocational 
teacher leaders. 
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EXECUTIVE SUMMARY 



This Manual is intended for use by persons who wish to learn more about the 
Leader Effectiveness Index {LEI), It contains (1) the conceptualization which provides the 
rationale for the instrument, (2) an explanation of how the instrument is to be used, (3) a 
report of the instrument's development and psychometric characteristics, and (4) a 
description of the process used to establish an appropriate norm group. 

In the conceptualization, the process of leadership is defined as "the use of 
noncoercive influence to direct and coordinate the attributes of the members of an organized 
group toward the accomplishment of group objectives" (Jago, 1982, p. 315). More 
specifically, this process is envisioned to achieve the following six broad tasks: (1) inspire 
a shared vision and establish standards that help the organization; (2) foster unity, 
collaboration, and ownership, and recognize individual and team contributions; (3) exercise 
power effectively and empower others to act; (4) exert influence outside of the organization 
in order to set the right context for the organization; (5) establish an environment conducive 
to learning; and (6) satisfy the job-related needs of members of the organization as 
individuals. Two studies are reported in the Manual which show that vocational teachers 
use the foregoing six broad tasks more than any others as criteria for evaluating the 
effectiveness of their administrator's performance as a leader. 

The LEI provides a multi-observer assessment of the effectiveness of leadership 
performance in vocational education. The 1993 edition contains seven items. The first six 
items measure the extent to which each of the above six leadership tasks are being 
performed; the seventh item measures the overall effectiveness of the leader's performance. 
A 6-point response scale follows each of the seven items. The scale (scored 1-6) ranges 
from "Not Effective" to "Extremely Effective"; a response of "Not Applicable" is also 
permitted. 

The test-retest correlation coefficient of the average rating of the six tasks, measured 
one week apart, is r=.94; the test-retest correlation coefficient of item 7 (overall 
assessment) is r=.95. Using the average ratings of three to five observers for each ratee, 
the internal consistency (Cronbach's alpha) of the first six items on the LEI is .92. The 
interrater reliability of the LEI, which measures the extent to which a group of three to five 
raters agree on their ratings of each ratee, is .86. 
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There are two common applications of the LEI. The first is a point-in-time 
assessment. The second is to measure change in leadership performance over time. A 
point-in-time assessment is the first step in leadership development. It sensitizes the ratee 
to the views of co7Workers about her or his performance as a leader, and it provides the 
motivation for improving performance and/or builds self-confidence. As a part of a point- 
in-time assessment, the LEI can be used in conjunction with the Leader Attributes 
Inventory (M/). After utilizing the LEI to ascertain the effectiveness, the ratee can use the 
LAI to assess the extent to which multiple observers believe she or he possesses each of the 
37 leader attributes. These attributes are characteristics, knowledge, skills, and values 
which have been found to be highly related to performance as a leader in vocational 
education. Individualized feedback from the LAI will enaole the ratee to identify the 
specific attributes that should be strengthened in order to improve leadership performance 
(Moss, Lambrecht, Finch, & Jensrud, in press). 

Measuring change in leadership performance over time can be accomplished by 
administering the LEI twice — once as a pretest and again as a posttest (using the usual 
experimental design precautions). For example, the effectiveness of a leadership 
development program can be measured by having each program participant use the LEI to 
secure ratings about her- or himself from three to five observers. After the program has 
been completed, and sufficient time has elapsed for any behavioral change to become 
evident, the same three to five observers can rate a participant a second time. 

For the purpose of establishing comparison (norm) groups for use with the LEI, 
vocational educators with three different roles were used to form three purposive samples: 
(1) chief vocational administrators (CVA), (2) vocational department heads (VDH), and (3) 
vocauonal teacher leaders (VTL). The samples were drawn from 12 states, each deemed to 
have a strong secondary or postsecondary vocational education system. LEI scores were 
collected from three to five subordinates and/or peers about each of 55 1 members of the 
three samples. The sample sizes were 220 CVAs, 168 VDHs, and 163 VTLs. Since no 
significant differences were found among the mean scores of the three samples, they were 
combined to form one norm group (n=551) entitled "Vocational Administrators and 
Vocational Teacher Leaders." 

Finally, utilizing the data about the norm group, the Manual provides directions for 
preparing and an example of an individualized feedback report. The report shows 
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graphically the average rating assigned to the ratee by three to five of his or her 
subordinates and/or peers, and a comparison of the average rating given the ratee with the 
average ratings given to the norm group of "Vocational Administrators and Vocational 
Teacher Leaders." 
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INTRODUCTION 



The Leader Effectiveness Index {LEI) is a multirater instrument designed to assess 
the effectiveness of leadership performance of vocational educators. It consists of seven 
items. The first six items are statements of six broad tasks (or responsibilities) of a leader 
in vocational education. The seventh item asks, "Overall, how effective is the leadership 
performance of the person you are rating?'* Each of the seven items is followed by a 6- 
point response scale. One possible response is a zero, "Not Applicable"; the other 6 points 
on the response scale range from "Not Effective" to "Extremely Effective." The average of 
the rating (1-6 points) on the first six items represents the perceived effectiveness of the 
leadership performance of the person being rated. Item 7 is included for research purposes 
(as in the validity study reported on p. 25). Appendix A contains a copy of the LEL 

The LEI is a companion instrument to the Leader Attributes Inventory {LAI) (Moss, 
Lambrecht, Finch, & Jensrud, in press). While the LEI measures the effectiveness of 
leader performance, the LAI provides a diagnostic assessment of the attributes that 
predispose desirable performance as a leader in vocational education. 

Background 

The LEI (as well as the LA/) has been developed over the past six years with 
funding from the National Center for Research in Vocational Education, NCRVE consists 
of seven institutions of higher education headed by the University of California at 
Berkeley. The other institutions are Teachers College at Columbia University, RAND 
Corporation, University of Illinois, University of Minnesota, University of Wisconsin, and 
Virginia Polytechnic Institute and State University. NCRVE is supported by a grant from 
the Office of Vocational and Adult Education, U.S. Department of Education, as authorized 
by the Carl D. Perkins Vocational Education Act. The grant is for conducting research, 
development, dissemination, and training that will improve the practice of vocational 
education in the United States. 

The Center's Role in Leadership Development 

NCRVE' s interest in leadership and leadership development stems from three 
sources. First, the Perkins Act requires that NCRVE **provide leadership development 
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services to vocational educators." The language of the Perkins Act, however, gives no 
clues about the way i:i which the requirement is to be satisfied. Second, persons 
throughout the country who were consulted about NCRVE's overall program of work, as 
well as those who were interviewed specifically for the purpose of exploring strategies for 
leadership development, agreed unanimously that vocational education does not now have 
the number of effective leaders that are urgently needed. More importantly, they also 
agreed that a systematic effort to develop leaders was not being made. Third, leadership 
becomes especially critical to organizations in unstable situations — situations in which 
change in the environment makes familiar ways of conducting the affairs of the 
organization unsatisfactory or irrelevant. NCRVE's Board of Directors believes strongly 
that, as much as in any previous era, vocational education is in just such a situation now. 
The field is faced with a series of changes that are rapidly and significantly altering the 
educational and economic environment in which it exists — changes in the nature of work, 
changes in the ethnic/cultural composition of the student body, and increasing public 
demands upon the education system. Vocational education must begin its own 
transformation if it is to remain a viable form of education in the new environment. 
Leaders are needed who can point to new directions and who can influence others to 
believe and to follow. 

The Status of Research About Leadership 

A great deal of research about leadership has been conducted during the last four 
decades in a wide variety of disciplines and fields of practice: philosophy, anthropology, 
psychology, sociology, political science, social psychology, management, and the military 
have all contributed to the body of literature. Education is a latecomer to the study of 
leadership and almost no research has been done in vocational education. Despite this 
considerable attention, it seems fair to say that, as yet, there is no consensus on a specific 
definition of leadership, an explanatory model of leadership behaviors, or the most useful 
means for measuring the effectiveness of leaders. There is, however, substantial 
agreement that leadership is a viable constiuct and that it can be recognized in practice, that 
aspects of leadership behavior can be measured and shown to be related to effective 
performance, and that educational interventions can effect the behavior of leaders. 
Summing up the progress made in the study of leadership since World War II, Kenneth 
Clark (1988) puts it this way: 

We may not havfe given the world a comprehensive theory of leadership, 
complete with knowledge about how to increase the quality and number of 
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leaders in future generations, but we have learned an enormous amount 
about the importance of certain qualities, about the effects of certain 
corporate or societal policies, and about ways in which persons with 
selected talents can be identified, (p. 1) 

NCRVE's Program of Work 

Given, on the one hand, NCRVE's compelling need to provide leadership 
development services for vocational educators, but faced, on the other hand, v/ith the 
absence of an agreed-upon comprehensive theory of leadership, NCRVE's first task 
became the creation of its own conceptualization of leadership and leadership development. 
The requirements of the conceptualization were that it be consistent with the results of prior 
empirical research and that it serve as a foundation for designing leadership development 
services and evaluating their effectiveness. 

The conceptualization that resulted from an extensive review of the literature, as 
well as interviews with leadership theorists and trainers, defines leadership and leadership 
development. It advances an explanation of the sources of leadership behavior, makes 
explicit the criteria for assessing leadership performance in vocational education, and 
hypothesizes a set of attributes — characteristics, knowledge, skills, and values possessed 
by individuals — which predispose desirable leader behaviors (Moss & Liang, 1990). 
Because the justification for the content and use of the LEI, as well as the LA/, resides in 
the conceptualization, its most relevant ideas are jsented in the next part of this section. 
The research that has been conducted to support the development of the LEI is presented in 
the third and fourth sections of this Manual. 

Concurrent with the creation of the LEI and the LA/, NCRVE has carried out a 
preliminary review of instructional materials used in leadership programs to identify those 
that are available, relevant, high quality, and low cost (Finch, Gregson, & Reneau, 1992). 
As a direct result of this review, NCRVE personnel created a series of case studies (Finch 
et al, 1992) and an administration simulation (Finch, 1992) that may be used in leadership 
programs to apply leader attributes in problem-solving and decision-making situations that 
are realistic to vocational educators. 

NCRVE has also stimulated, facilitated, and then evaluated the conduct of 17 new 
leadership development programs in universities across the country. The participants often 
of the programs were graduate students majoring in vocational education (Moss, Jensrud, 
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& Johansen, 1992); the participants of seven programs were inservice vocational teachers 
and administrators (Leske, Berkas. & Jensrud, 1993). The LA/ proved to be useful as one 
of the tools for assessing program effects. The results of the evaluations provided insights 
that have most recently been used by NCRVE personnel to create a new leader d;.velopment 
program for underrepresented groups in vocational education (Moss, Schwartz, & Jensrud, 
in press). 



Conceptualizing Leadership and Leadership Development 
A Definition 

From NCRVE' s perspective, leadership may be thought of 

as both a process and a property. The process of leadership is the use of 
noncoercive influence to direct and coordinate the attributes of the members 
of an organized group toward the accomplishment of group objectives. As 
?i property, leadership is a set of qualities or characteristics attributed to 
those who are perceived to successfully employ such characteristics. (Jago, 
1982, p. 315) 

Leadership, then, is the process of perceiving when change is needed and 
influencing the group by such noncoercive means as persuasion and example in its efforts 
toward goal setting and goal achievement. 

Tat property of leadership is ascribed to an individual by members of a group when 
they perceive the individual (inferred from her or his behavior) to possess certain qualities 
or characteristics. Members of the group allow an individual to lead— to influence them— 
when the individual's behaviors match the group's ideas about what good leaders should 
be and do in that context. Since leadership as a property lies in the eye of the beholder, 
only those who are so perceived are leaders. The specific properties of leadership depend 
upon the qualitative nature of the behaviors accepted by a particular group as evidence of 
leadership. Given this concept, the perceptions of potential followers— subordinates or 
peers in formal organizations — are of primary importance when assessing the effectiveness 
of leadership. 

Individuals who are seen as leaders enjoy the power of influence that is voluntarily 
conferred (Gardner, 1986b). By contrast, individuals appointed to supervisory positions 
within organizations— for example, as head or administrator — have the power of authority 
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as a result of holding their positions. However, although supervisors can be given 
subordinates, they cannot be given followers. They must earn followers by displaying the 
qualities their subordinates ascribe to leadership. Consequently, any individual in the 
vocational education community—teacher, counselor, and administrator— can demonstrate 
behaviors consistent with the properties of leadership and be considered a leader by the rest 
of the group. While administrative positions in organizations may offer more opportunities 
to demonstrate leadership than some other positions, the position itself does not 
automatically confer leadership upon the holder. 

Vocational education institutions and agencies (and the vocational education 
enterprise as a whole) must have leaders at all levels and in all professional roles. Certainly 
it is critical for top-level administrators to be good leaders; in order for organizations to 
achieve peak efficiency, however, leaders are needed throughout the organization (and the 
profession) in positions that have no authority as well as those that do. 

The Tasks 

The process of leadership may be further elaborated and the concept of leadership 
better understood by describing the broad tasks that comprise the leader's expected role in 
organized groups. 

The perspective taken by NCRVE is that the leader's aim should be to achieve 
maximum group productivity (Cleveland, 1985; Kanter, 1981) by bringing into focus the 
organization's vision, mission, and values; helping to adapt the organization to the 
environment; and securing the commitment of individuals in the organization and fostering 
their growth by tapping their intrinsic motivation. This conception of a leader's role is 
essentially one of facilitating the group process and empowering group members through 
the use of consultation, persuasion, and inspiration. As John Gardner (1986a) has so 
eloquently stated. 

Perhaps the most promising trend in our thinking about leadership is the . 
growing conviction that the puiposes of the group are best served when the 
leader helps followers to develop their own initiative, strengthens them in 
the use of their own judgment, [and] enables them to grow and to become 
better contributors. [The problems we face] simply cannot be dealt with 
unless there are highly motivated workers who are accustomed to taking 
responsibility. 
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To the extent that leaders enable followers to develop their own initiative, 
they are creating something that can survive their own departure, (p. 23) 

In order to translate NCRVE's perspective into more specific criteria that can be 
used to evaluate a leader's performance, four broad leadership tasks were first synthesized 
from several sources (Bass, 1981; Gardner, 1987; Posner & Kouzes, 1988; YukI & Van 
Fleet, 1982). Two tasks philosophically consistent with the first four were added later as 
the result of a validation study conducted to determine the tasks that are actually used as 
criteria by vocational teachers when they evaluate the leadership performance of their 
administrators (Moss, Finch, & Johansen, 199 1).^ The six leader tasks that describe the 
envisioned role of leaders in vocational education and which serve as criteria for the 
measurement of leader performance are as follows: 

1 . Inspures a shared vision and establishes standards that help the organization achieve 
its next stage of development. For example, creates a sense of purpose, defines 
reality in the larger context, and instills shared values and benefits. 

2. Fosters unity, collaboration, and ownership, and recognizes individual and team 
contributions. For example, creates a climate of community, builds morale, sets a 
positive tone, and resolves disagreements. 

3. Exercises povv^er effectively and empowers others to act. For example, facilitates 
change, shares authority, and nurtures the skills of group members. 

4. Exerts influence outside of the organization in order to set the right context for the 
organization. For example, serves as a symbol for the group, secures resources, 
builds coalitions, and acts as an advocate. 

5. Establishes an environment conducive to learning. For example, provides 
intellectual stimulation creates a supportive climate for learners, and facilitates the 
professional development of staff. 

6. Satisfies the job-related needs of members of the organization as individuals. For 
example, respects, trusts, and has confidence in members, adapts leadership style 
to the situation, and creates a satisfying work environment. 



The study is more fully described in Section 3. 
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Leader Behaviors 

As leaders attempt to achieve the six tasks of leadership, their specific behaviors 
within an organization are determined by their own attributes (which Jago calls qualities) 
interacting with then- perception of (1) the group members' attributes (including the group's 
culture), (2) the particular task at hand, and (3) the general context in which the 
organization is operating. The behaviors that stem from this interaction are very situational; 
they change with the leader's perception of the prevailing context, the immediate task, and 
the relevant quaUties of the group. 

Group members filter the leader's behavior through their own perceptions of the 
context, the task at hand, and the leader's attributes, and then behave within the constraints 
of their own attributes. The meaning systems of the leader and the group must, therefore, 
correspond or the intent of the leader's behavior will be misunderstood. 

This model is depicted in Figure 1-1. Two feedback loops are shown. First, the 
leader may adjust perceptions of her or his own attributes or the group's attributes or both, 
and may even make a mid-course change in behavior, as a result of group behavior. 
Second, members of the group may adjust their perceptions of the leader's or their own 
attributes or both, and may even adjust their behavior in mid-course, as a result of their 
assessment of the leader's behavior. One implication of the model is that the leader is 
influenced by the group, as well as vice versa, thereby making leadership behavior a 
possible dependent as well as an independent variable. 

Several classification systems have been created for categorizing the behaviors of 
leaders. One system classifies behavior as either initiating structure (task-oriented 
behaviors) or initiating consideration (people-oriented behaviors). Research has shown 
that the most effective leaders exhibit behaviors in both categories, with the balance 
influenced by the nature of the group, the task, and the context. More recently, behaviors 
have been categorized as either transactional or transformational (Bass, 1985; Bums, 
1978). Transactional leaders give something in exchange for what they want, they direct 
energy, tend to live within the organizational culture, and hold followers in a dependent 
position. By contrast, transformational leaders synergize the energy of followers, alter the 
culture, and put themselves and their followers in an interdependent relationship. 
Researchers agree that both transactional and transformational behaviors are needed to 
accomplish the broad tasks of leadership. It has been shown, however, that leaders who 
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exhibit greater amounts of transformational behaviors have a more positive impact on such 
criteria as team performance, subordinate's evaluation of effectiveness, satisfaction with 
leaders, and supervisor's ratings of leader performance (Avolio & Bass, 1988; Bass, 1987; 
Clover, 1988; Hater & Bass, 1987; Mueller, 1980; Yammarino & Bass, 1988) 
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Leader Attributes 

While it is a leader's behaviors that directly influence group performance, it is a 
leader's attributes— the characteristics, knowledge, skills, and values possessed by the 
leader— that shape those behaviors. Within the constraints of a given situatioj^, attributes, 
acting as predispositions, disinhibitors, and abilities, predispose individuals to behave in 
consistent ways. Attributes remain constant across situations to influence behavior in a 
wide array of tasks, groups, and contexts (Lord, DeVader, & Alliger, 1986), For 
example, attributes determine the tendency of an individual to use transactional or 
transformational behaviors (Brown & Hosking, 1986; Kuhnert & Lewis, 1987), The 
greater the latitude provided by the situation, the more likely it is that attributes will shape 
and guide behavior. Bass (1981) sums up this way: 

Strong evidence has been found supporting the view that leadership is 
transferable from one situation to another. Although the nature of task 
demands may limit transferability, there is a tendency for the leader in one 
group to emerge in this capacity in other groups, (p. 596) 

Many researchers have linked attributes directly to an array of leadership criteria in a 
wide variety of situations. For instance, the kind and the amount of certain attributes an 
individual possesses have been shown to be consistently and significantly related to such 
measures as rated managerial performance, advancement in business and education, and the 
emergence and succession rate of leaders (Arter, 1988; Behling & Champion, 1984; 
Hogan, Raskin & Fazzini, 1988; Hollander & Offermann, 1988; House, 1988; Sashkin & 
Burke, 1988; Yukl, 1981). Earlier reviews of the literature— for example, Stogdill 
(1948) — are often thought to have revealed that there are no relationships between 
intelligence, personality factors, and leadership. More recently. Lord et al. (1986) have 
used current meta-analysis methods to show that, to the contrary, the literature has shown 
that therei are significant and consistent relationships between personality factors and 
intelligence and the emergence of leadership. 

It can be presumed, then, that there are some attributes, which, if possessed in 
adequate amounts, will increase the likelihood that desirable leadership behaviors will occur 
in a wide variety of situations, particularly if those situations occur within a limited general 
context such as vocational education. 

What are those specific attributes? Although research on leaders and leadership in 
vocational education is almost nonexistent, the literature of several other fields is filled with 
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nominations based upon theory, experience, and empirical research. In one publication 
alone, Bass (1981) reviewed 124 studies completed between 1904-1947 and 215 more 
between 1947-1970. Although no two studies were found to advance exactly the same set 
of attributes, there is a great deal of consistency among the kinds of attributes proposed. 
After reviewing a large number of available publications and interviewing several 
leadership theorists and trainers, a list of 37 attributes was compiled.^ The list consists of 
the attributes that are hypothesized to predispose the behaviors that will achieve the six 
broad tasks of leaders in vocational education. They are shown in Table 1-1 and are 
assessed by the Leader Attributes Inventory. 

It has been assumed that the amount of each attribute possessed by individuals is 
normally distributed in the population of vocational educators. While some of the 37 leader 
attributes may be quite resistant to improvement, prior research has demonstrated that some 
of the attributes common to successful leaders can be increased by a reasonable amount of 
planned educational experiences (Bass, 1981; Lester, 1981; Manz & Sims, 1986; 
Yammarino & Bass, 1988; Yukl, 1981). The objective of leadership development activities 
should, therefore, be to improve those leader attributes that prove to be susceptible to 
change by educational interventions. Those attributes that prove to be resistant to change 
could provide a basis for selecting high potential trainees. 




2 Originally, the list consisted of 35 attributes. In the process of developing the Leader Attributes 
Inventory (LAI), one of the attributes was found to be best stated as two attributes, and one new attribute 
was added, bringing the total to 37. 
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Table 1-1 

Attributes Assessed by the Leader Attributes Inventory {LAI) 



Attribute 


1. 


Rnerpetic with ^tflmina 


on 
zu. 


i:!«uuC3i 


2. 


Insiffhtful 


0 1 


Communication (listening, oral, 


3 






written) 


4. 


Visionarv 


zz. 


oensiiivity, respect 


5. 


Tolerant n"f PmllianitV JjnH r»rvmnlpYit\/ 


0'^ 
Z J. 


Motivating others 


6. 


Achievempnt-nripntpH 


' 0/1 


Networking 


7. 


AccountAhle 


0^ 
Z J. 


jriannmg 


8. 


Initiating 




J-AJiegdUIlg 


9. 


Confident accenting of <;plf 


07 
z / . 


organizing 


10. 


Willin? to accent rp<;non<;ihilitv 


OR 
zo. 


I earn DUiiaing 


11. 


Persistent 


00 


Coaching 


12. 


Entliusiastic, optimistic 




v-^umuci management 


13. 


Tolerant of frustration 


31. 


Time management 


14. 


Dependable, reliable 


32. 


Stress management 


15. 


Courageous, risk-taker 


33. 


Appropriate use of leadership styles 


16. 


Even disposition 


34. 


Ideological beliefs appropriate to the 


17. 


Committed to the common good 




group 


18, 


Personal integrity 


35. 


Decision-making 


19. 


Intelligent with practical judgment 


36. 


Problem-solving 






37. 


Information management 



The Objectives of Leadership Development 

The general purpose of leadership development in vocational education is, 
therefore, to increase the number and quality of leaders prepared to meet present and future 
challenges facing the field. More specifically, NCRVE seeks to accomplish that purpose 
by deliberately attempting to effect positive change in selected attributes (characteristics, 
knowledge, skills, and values possessed by individuals) to increase the likelihood that 
vocational educators will (1) perceive opportunities to behave as leaders, (2) grasp those 
opportunities, and (3) succeed in achieving the six tasks of leaders in a wide variety of 
situations and professional roles. 



Leadership Development as a Part of Professional Development 

Leadership development attempts to cultivate selected attributes to enhance the 
probability of successful performance as a leader in a wide variety of situations. These 
attributes are common to leadership behavior in all professional roles in vocational 
education; administrators should have them, teachers should have them, and counselors 
should have them. But in order to perform successfully as administrators or teachers, 
individuals need more than the common leadership attributes. They also need the 
knowledge and skill attributes that are unique to their given roles. These are the attributes 
that distinguish administrators from teachers, teachers from counselors, and counselors 
from administrators, and that determine whether individuals can perform the specific 
occupational or technical tasks of their professional roles. Leadership development is, 
thus, only one part of professional development. Professional development consists of 
cultivating both the leadership attributes and the attributes that facilitate successful 
performance in a particular professional role. 
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USING THE LEADER EFFECTIVENESS INDEX 



Description 

The Leader Effectiveness Index (LEI) provides a multi-observer assessment of the 
effectiveness of leadership performance in vocational education. The 1993 edition contains 
seven items. The first six items measure the extent to which each of six leadership tasks 
(responsibilities) are being performed; the seventh item measures the overall effectiveness 
of the leader's performance. A 6-point response scale follows each of the seven items. 
The scale (scored 1-6) ranges from "Not Effective" to "Extremely Effective"; a response of 
"Not Applicable" is also permitted. (See Appendix A.) 

The version of the LEI contained in Appendix A does not ask for any information 
about the observer-raters, Questions appropriate to each situation such as gender, ethnic 
group membership, position, how well the ratee is known, and so on, can be collected 
separately, or added to the instrument itself, by the person managing the data collection 
process. Provision is made on the LEI, however, for entering an identification number for 
each rater. This eliminates the need to obtain the name of the observer-rater, thereby 
helping to preserve confidentiality while permitting the rating to be identified with a 
particular ratee. 

The instrument can be completed in less than five minutes. In order to balance the 
demands of efficiency and interrater reliability, and to acquire valid ratings, three to five 
subordinates (or peers if there are not enough subordinates) who know the ratee well at 
work should be used as observer-raters for each ratee. To maintain the confidentiality of 
ratings (and therefore their candor), all of the completed forms should be returned directly 
to the LEI vendor for scoring and completion of feedback reports. 

The LEI produces a single score for each ratee. It is the average rating on items 1- 
6, itself averaged over the number of observer-raters used for each ratee. 

Individualized feedback reports to the individuals being rated contain two charts 
which present the ratee' s average obser\'er-rating and compare the ratee' s average observer- 
radng with the average observer-ratings of a norm group of vocational administrators and 
vocational teacher leaders. Appendix B presents a sample of a feedback report that contains 
the two charts and an explanation of how the feedback is to be interpreted. The section 
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entitled, "Establishing Norms and Standards for the LEI'' provides a description of the 
vocational norm group presently available for comparison and how it was created. 

Applications 

There are two common applications of the LEL The first is a point-in-time 
assessment. The second is to measure change in leadership performance over time. 

A point-in-time assessment is the first step in leadership development. It sensitizes 
the ratee to the views of co-workers about her or his performance as a leader, and it 
provides the motivation for improving performance and/or builds self-confidence and 
"psychological capital" (probably with attendant improved performance). As a part of a 
point-in-time assessment, the LEI can be used in conjunction with the Leader Attributes 
Inventory {LAI). After utilizing the LEI to ascertain the perceptions of co-workers about 
his or her leadership effectiveness, the ratee can use the LAI to assess the extent to which 
multiple observers believe she or he possesses each of the 37 leader attributes. These 
attributes are characteristics, knowledge, skills, and values which have been found to be 
highly related to performance as a leader in vocational education. Individualized feedback 
from the LAI will enable the ratee to identify the specific attributes that should be 
strengthened in order to improve leadership performance (Moss, Lambrecht, Finch, & 
Jensrud, in press). 

Measuring change in leadership performance over time can be accomplished by 
administering the LEI twice — once as a pretest and again as a posttest (using the usual 
experimental design precautions). For example, the effectiveness of a leadership 
development program can be measured by having each program participant use the LEI to 
secure ratings about her- or himself from three to five obse'A^er-raters. After the program 
has been completed, and after sufficient time has elapsed for any behavioral changes to 
become evident, the same three to five observer-raters can rate the participant a second 
time. 
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Supporting Materials 

In January 1994 a team of developers supported by the National Center for 
Research in Vocational Education completed a leadership program entitled Preparing 
Leaders for the Future: A Developmental Program for Underrepresented Groups in 
Vocational Education (Moss, Schwartz, & Jensrud, in press). The program consists of 32 
learning experiences (lessons) requiring approximately 90 clock hours of in-class 
instruction plus out-of-class assignments. The program may be adapted to various lengths 
and to suit the developmental needs of specific groups and individuals. 

One section of the program is designed to assist participants in planning for their 
leadership development. Included in that section are two learning experiences which (1) 
introduce the conceptualization of leader attributes and (2) use the LAI self-rating and 
rating-by-observer forms to produce an assessment of each participant's leader attributes. 
A third learning experience in the same section uses the results of the attribute assessment, 
together with the results of other instruments, to help each participant formulate a 
leadership development plan. 

Another section of the program is designed to develop participants' leader 
attributes. Twenty-four learning experiences are provided to improve participants' 
performance on 23 of the 37 attributes assessed by the LAL Each learning experience is 
focused on an attribute that is presumed to be "improvable" through a planned learning 
experience. The final learning experience requires the application of all the attributes 
through a simulation exercise in which program participants administer a large 
postsecondary institution (Finch, 1992). 

Each learning experience in the program contains the following: (1) a statement of a 
performance objective, (2) a description of the steps in the process and the knowledge base 
required to perform the desired behavior(s), (3) a plan for delivering the learning 
experience, (4) an outline of the content to be covered by the learning experience, (5) out- 
of-class assignments, and (6) master copies of the handouts and transparencies to be used 
in the delivery of the learning experience. 
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DEVELOPING THE LEADER EFFECTIVENESS INDEX 

Introduction 

As conceived by NCRVE, the purpose of leadership development is to effect 
positive change in selected attributes (characteristics, knowledge, skills, and values 
possessed by an individual) to increase the likelihood that vocational educators will (1) 
perceive opportunities to behave as leaders, (2) grasp those opportunities, and (3) succeed 
in achieving the tasks of leaders in a wide variety of situations and professional roles. The 
role of leaders in vocational education is perceived primarily as facilitating the group 
process and empowering group members. Leaders carry out this role by performing the 
following six broad tasks: (1) inspire a shared vision and establish standards that help the 
organization achieve its next stage of development; (2) foster unity, collaboration, and 
ownership, and recognize individual and team contributions; (3) exercise power effectively 
and empower others to act; (4) exert influence outside of the organization in order to set the 
right context for the organization; (5) establish an environment conducive to learning; and 
(6) satisfy the job-related needs of members of the organization as individuals. The Leader 
Effectiveness Index {LEPj has been developed to measure the extent to which vocational 
education professionals are achieving these six tasks. This section describes the steps in 
the development of the LEL 



Developmental Process 

Initially, the LEI contained only the first four of the six broad tasks listed above 
(Liang, 1990; Moss & Liang, 1990). The four tasks, as four items with a 5-point response 
scale ranging from "Extremely effective" to "Not effective," were appended as a separate 
section of the Leader Attributes Inventory {LAI). As a part of the Liang study, a subsample 
of 36 postsecondary vocational teachers were asked to rate two times, with an interval of 
two weeks between ratings, the vocational administrator whom each knew best. The test- 
retest reliability coefficient of the total score of the four items was .92. 

In 1990, a class of 38 master's- and baccalaureate-level students majoring in 
management completed the next version of the LEL This version continued to include the 
same four tasks used in the Liang study, but added a fifth task, "plays the political role." 
The same 5-point response scale was utilized. Since all of the students were employed in 
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business and industry, they were asked to rate their present supervisors on the five items 
and on the LAL The LEI was administered twice three weeks apart. The test-retest 
correlation of the total LEI score was .90. The total score of the five LEI items and the total 
score of the first four items (excluding **plays the political role") were then correlated with 
each of the 37 attributes on the LAL The resulting coefficients were identical. It was 
concluded that the fifth task-item, **plays the political role," did not contribute any 
dimensions to the leadership role that were not already being assessed by the first four LEI 
task-items. 

It was also recognized in 1990 that the type of outcomes or criteria that group 
members might utilize when they judge leader effectiveness can differ widely, and that it 
was critical to determine whether the criteria actually used by vocational educators were 
those reflected in the four broad tasks being used by NCRVE. In other words, was 
NCRVE's view about the facilitating and empowering role of effective leaders the view 
used by vocational educators as they judged theu* leaders? 

Three types of criteria for judging the effectiveness of leaders are found in the 
literature (Yukl, 1989). The most common type is the extent to which the leader's group or 
organization performs its tasks successfully or reaches its goals (organizational outcomes). 
A specific example of this type, assessed subjectively, might be the degree to which 
members of an organization ascribe its success to the efforts of the leader. Another 
common outcome is the personal impact of leaders on followers (impact on instructors). A 
subjective measure of this type might Le the expressed strength of the followers' 
conmiitment to carry out the leader's requests. A third type of outcome is the leader's 
contribution to the quality of the group process (group process). For example, to what 
degree is the leader perceived to enhance group cohesiveness and member motivation (the 
facilitative, empowering role)? 

A system for classifying leadership effectiveness criteria was created by Moss, 
Finch, and Johansen (1991) to reflect the three types of outcomes proposed by Yukl 
(1989). For each of the three types of outcomes, categories of criteria were created. 
Examples of specific criteria were then specified to illustrate the types of outcomes that 
might be included as a part of each category. Table 3-1 shows the three types of outcomes 
and the categories of criteria that were used for each type. Note that the five categories of 
criteria used with the type of outcome **Group Process," that is, facilitating the group 
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process and empowering group members, included the first four tasks that had previously 
been used as criteria on the LEI. A fifth category (or task), "establish a learning 
environment," was added to reflect a specialized category of outcomes appropriate to the 
educational context in which leaders in vocational education perform. 

In order to validate these leader effectiveness criteria, vocational instructors were 
asked to identify events describing the behavior of successful vocational administrators. 
Data was collected from a purposive sample drawn from seven states, each with a well- 
developed system of secondary or postsecondary vocational education (Finch, Gregson, & 
Faulkner, 1991). In each state, the chief state administrative officer for vocational 
education and his or her inmiediate subordinates were interviewed to identify the most 
successful administrators of specialized vocational institutions— area vocational schools, 
vocational centers, and technical colleges. The two to seven local administrators nominated 
most frequently by the state staff were then contacted and asked to take part in the study. A 
total of 39 chief administrators of specialized vocational institutions (all of those invited) 
agreed to participate. Twenty participants administered secondary schools and 19 
administered postsecondary schools; only three were females. 
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Table 3-1 

Frequency of Use by Category of Criteria and Gender 



Category 


Female 
n % 


Male 
n % 


Total 
n % 


Group Process 




IT * 1 1 « « 

1. Inspire a shared Vision 


1 


0 


3 


1 


4 


2 


2. Achieve unity and motivate 


15 


6 


13 


6 


28 


12 


3 . Implement change and power 


21 


9 


19 


8 


40 


. 17 


4. Exert external influence 


18 


8 


15 


6 


33 


14 


5 . Estabush learmng environment 


15 


6 


9 


4 


24 


10 


Impact on Instructors 




1 . Satisfy job-related needs 


20 


8 


27 


11 


47 


20 


2. Increase engagement with work 


4 


2 


4 


2 


8 


3 


wrganizaiionai uutcomes 




I . Improve instruction 


9 


4 


13 


6 


22 


9 


2. Provide equity and access 


6 


3 


2 


1 


8 


3 


3 . Increase labor market responsiveness 


14 


6 


5 


2 


19 


8 


4. Satisfy student development 


3 


1 


1 


0 


4 


2 


Totals 


126 


53 


111 


47 


237 


100 



Each of the 39 administrators was asked to provide the names of six instructors in 
his or her school. Two of the six instructors were then randomly selected by the 
investigators so as to balance gender and occupational field. The 78 instructors chosen (39 
administrators chose 2 instructors each) represented all of the vocational service areas as 
well as the related academic subjects. 

The instructors were sent a letter requesting their cooperation in the study and 
telling them the kinds of questions they would be asked. Semi-structured telephone 
interviews were then conducted with all 78 instructors. As a part of the interview, each 
instructor was asked to recall two incidents or events in which her or his administrator was 
particularly effective as a leader and to provide a very detailed description of each event. 



Each interview was recorded and transcribed and then the interviewer completed a 
write-up of each event. The purpose of the interview write-up was to organize and present 
the interview transcript and note-taking information in a more easily understandable format. 
Information contained in the write-up was organized into sections on ^'situation," "who was 
involved," "behavior," "thoughts/feelings," "outcome," and "writer comments'' 
(Mentkowski, O'Brien, McEachem, & Fowler, 1982). All the transcripts and write-ups 
were reviewed by a second person to assure their accuracy and completeness. The write- 
ups of 154 events (two instructors provided descriptions of only one event each) comprised 
the database. 

Each write-up was analyzed to determine the category or categories of criteria that 
the instructor was implicitly using when he or she identified the event as a particularly 
effective leadership behavior (Moss, Finch & Johansen, 1991). Two of the investigators 
read all of the write-ups in groups of ten. No difficulties were encountered in "fitting" the 
events into the classification system. After they had read and classified each write-up in a 
group as representing one or more of the categories of criteria, the investigators met to 
compare results. When they found a difference in the classification(s) of a write-up, they 
discussed their reasons and reached accord. For the first 80 write-ups the average 
agreement of the investigators before discussion was 70%. For the last 74 write-ups, the 
agreement before discussion reached an average of 91%. 

As the investigators classified the write-ups, it immediately became evident that for 
many of the events there was a primary criterion being used as well as one or two 
additional (secondary) criteria. Instructors were often employing multiple categories of 
criteria to judge an event as evidencing particularly effective leadership. In some cases, the 
multiple categories of criteria were a part of the same type of outcome (division of the 
classification system) — for example, quality of the group process — but in 56 events the 
multiple categories of criteria were drawn from different types of outcomes, for example, 
quality of the group process and personal impact on followers. 

Since the interpretation of events revealed that many reflected the use of one or two 
secondary criteria as well as primary criterion, data analyses were conducted using two sets 
of data: (1) the primary criteria only and (2) combining all of the criteria and giving equal 
weight to primary and secondary criteria. There are theoretical advantages and 
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disadvantages to using each data set, but as the two analyses yielded identical results, only 
the analysis using the combined criteria is presented here. 
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Table 3-1 presents the frequencies and percentages with which instructors used the 
criteria to determine which behaviors represent effective leadership by administrators in 
vocational education. A number of chi-square analyses were conducted and, based on 
these tests, several conclusions were reached. First, in a hypothetical population like the 
sample in this study, vocational instructors use the type of outcome, "Extent to which the 
leader's behavior is perceived to improve the quality of the group process" more than either 
of the other two types of outcomes (X2=47.50; df=2; x2.o 1=9.21). Second, it was 
concluded that the gender of the vocational instructor was not related to the category of 
outcomes used (X2=10.98; df=10; X2 oi=23.21). Finally, it was concluded that there are 
significant differences among the frequencies with which vocational instructors used the 
categories of outcomes to assess effective leader performance (X2=100.0; df=10; 
X2.0l=23.21). Although "satisfy followers' (instructors') job-related needs and 
expectations" was the single most utilized category of criteria, four of the five categories in 
the "group process" type of outcome were ranked second to fifth in terms of their use. On 
the other hand, "inspire a shared vision" was tied for being used least frequently. Data 
fi-om the study provides no explanation for this unexpected finding. 

Based on the results of this study, it was decided to utilize six criteria (tasks) in the 
Leader Effectiveness Index (LEI): (1) inspire a shared vision and establish standards that 
help the organization achieve its next stage of development; (2) foster unity, collaboration, 
and ownership, and recognize individual and team contributions; (3) exercise power 
effectively and empower others to act; (4) exert influence outside of the organization in 
order to set the right context for the organization; (5) establish an environment conducive to 
learning; and (6) safisfy the job-related needs of members of the organization as 
individuals. The "visioning" task was included despite its low frequency of use as a 
criterion by vocational educators because it is consistent with the philosophic position of 
the NCRVE and because it is almost unanimously perceived by scholars as critical to what 
leaders should be accomplishing. The other five tasks were used most by instructors in the 
study; four of the five reflect improving the quality of the group process, which is 
consistent with the NCRVE's facilitative, empowering perspective of the leader's role. It is 
apparent that the vocational educators used in the sample also see the role of effective 
administrator-leaders as "empowerers" rather than as "controllers." 
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1993 Form 

The 1993 form of the LEI contains the six items/tasks listed above, plus a seventh 
item designed to measure the respondent's overall assessment of a leader's performance: 
"Overall, how effective is the leadership performance of the person you are rating?" A 6- 
point response scale is provided for the seven items, ranging from "Not Effective" to 
"Extremely Effective." A response of "Not Applicable" is permitted. A copy of the LEI is 
contained in Appendix A. 

Face Validity 

In every version of the LEI, with all of the groups in which they have been used, 
no respondents have reported that any of the tasks were irrelevant to their conception of 
leadership. This acceptability is considered evidence of the instalment's face validity. 

Construct Validity 

Two studies have been conducted to assess the instrument's construct validity. The 
first, reported fully in Moss, Finch, and Johansen (1991), has already been described 
earlier in this section. That study determined the criteria actually used by vocational 
teachers to assess the effectiveness of their administrator's performance as a leader. The 
results indicated that the vocational teachers in the sample endorsed NCRVE's 
conceptualization of the leader's role as primarily a facilitator of the group process and an 
empowerer of group members. In other words, the LEI permits its intended users to 
express their own beliefs (as well as NCRVE's beliefs) about the leaders in vocational 
education. 

In the second study, the 1993 edition of the LEI was administered to two groups of 
graduate students majoring in vocational education (n=37, n=38). The correlation 
coefficients between the average score of the first six items (the six broad tasks of leaders) 
and item 7 (overall assessment) of the two samples were r=.91 and .92. The average 
difference between the mean score of items 1-6 and item 7 was only ,054 (X 1,5=3. 027; 
X 7=2.973). Thus, with the average score of the six tasks practically the same as the score 
on item 7, and the correlation coefficient between them so high, the six tasks measured by 
the LEI appear to be assessing the complete set of criteria respondents used to judge leader 
effectiveness. This is a confirmation of the results of the study that showed the six tasks 
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measured by the LEI represent the criteria used by vocational educators when they judge 
the effectiveness of a leader's performance. 

Test-Retest Reliability 

In the second study noted above, the LEI was administered one week apart to each 
of the two groups of graduate students (n=37, n=38). The test-retest correlation 
coefficients of the average rating on the six tasks were r=.94 and .93. The test-retest 
correlation coefficients of item 7 (overall assessment) were r=.95 and .92. These 
coefficients demonstrate the satisfactory short-term stability of the LEI scores. 

Internal Consistency 

Internal consistency indicates the extent to which the items comprising the 
instrument are measuring the same concept. Cronbach's alpha is the statistic most widely 
used to assess internal consistency. 

The data from the study that was used to establish one or more norm groups for the 
LEI was also used to compute internal consistency (see the next section for a description of 
the samples and the norming process). Using the average of three to five ratings-by- 
observers over the first six LEI items for each ratee and a sample of 551 ratees, the alpha of 
the current version of the LEI was .92. 

Interrater Reliability 

Agreement within groups of raters is called interrater reliability. In this case, it 
measures the extent to which a group of three to five raters, each using his or her 
perception of the ratee and the first six LEI items, agree on their ratings of the ratee. The 
data from three to five raters about each of 551 ratees were collected as a part of the study 
designed to establish norms for the current LEI. (See the next section for a description of 
the norming study.) The raters were either subordinates or peers of the ratees (peers were 
used when the ratee did not have five subordinates). The interrater reliability of the average 
rating of the first six LEI items/tasks is .86. 




ESTABLISHING NORMS FOR THE LEI 



Establishing norms enhances the meaningful interpretation of LEI scores. 
Certainly, learning how co-workers feel about the ratee's leadership performance provides 
very useful information, but persons being rated also want to know, "How are my ratings 
relative to the ratings of others in my group (or the group to which I aspire)?" Research by 
Moss, Jensrud, and Johansen (1992) has demonstrated that knowledge about the strength 
of one's attributes, relative to an appropriate comparison group, motivates participants of 
leadership development programs to set meaningful personal improvement goals and to 
strive to attain them. Consequently, a study was conducted during 1993 to establish norms 
for the LEL 



Identifying the Samples 

Vocational educators with three different roles (each with an expectation that 
effective leadership should be provided) were used to form three purposive samples: 

1 . Chief Vocational Administrators ( CVA). These were the chief line administrators in 
(1) specialized public secondary vocational institutions, for example, principals and 
directors; and (2) both specialized and comprehensive public postsecondary 
institutions, for example, presidents, directors, and deans. 

2. Vocational Department Heads (VDH), These were administrators/managers of 
clusters of vocational programs,; for example, department heads and coordinators in 
specialized public secondary vocational institutions, and both specialized and 
comprehensive public postsecondary institutions. 

3. Vocational Teacher Leaders (VTL). These were professionals in non- 
adnainistrative/management positions, for example, teachers and counselors, who 
were viewed by their chief administrator and/or department head as particularly 
influential among their peers. Examples are teachers who held elective positions in 
faculty associations, professional organizations, or unions. 

With the advice of consultants, a group of twelve states was selected from which 
the three samples were drawn. These states were deemed to have well-developed 
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secondary or postsecondary vocational systems with relatively high proportions of minority 
teachers and administrators. Table 4-1 presents the total population of chief vocational 
administrators in each of the twelve states by type of institution. It is from this population 
that the sample of chief vocational administrators was selected. 

All of the 329 chief vocational administrators (CVA) in the twelve states were 
contacted by letter and then by telephone. The study and their role in it was explained, and 
their agreement to participate was solicited. Three hundred eleven CVAs agreed to take 
part. Whether or not the CVAs agreed to participate, they were also asked to nominate (1) 
three vocational department heads (VDH) including (where possible) at least one member 
of a minority group anu one female; and (2) up to three vocational teacher leaders, giving 
consideration to minorities and females whenever possible. 



Table 4-1 

Population of Chief Vocational Administrators 

(n-329) 



State 


Technical College 
Administrators 


Community College 
Administrators 


Secondary 
Administrators 


Arkansas 


24 






Colorado 




12 




Florida 




27 




Georgia 


32 






Illinois 




38 




Iowa 




18 




Maryland 






33 


Ohio 






60 


Oklahoma 






30 


Oregon 




13 




Tennessee 


26 






Wisconsin 


16 






Totals 


98 


108 


123 
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Vocational department heads (VDH) were then called and their participation in the 
study solicited. Minority VDHs were contacted whenever they had been nominated by 
CV As. When no minority member had been nominated, or if the nominee declined to take 
part in the study, an attempt was made to randomly select an equal number of men and 
women to contact for the VDH sample. Two hundred eighty-nine vocational department 
heads consented to take part in the study. During the interviews, VDHs were asked to 
provide the names of up to three vocational teacher leaders, giving consideration to 
including minorities and females whenever possible. 

Finally, vocational teacher leaders (VTL) were called and invited to be a part of the 
study. Minority members who had been nominated by either the CVA or VDH at a given 
institution were called first. If no minority member had been nominated, or if she or he 
refused to participate, individuals who had been named by both the CVA and the VDH 
were called. In lieu of joint nominees, an attempt was then made to randomly telephone an 
equal number of men and women. A total of 305 vocational teacher leaders agreed to 
participate in the study. 



Collecting the Data 

Each of the 905 persons who consented to take part in the norming study was sent a 
packet of materials containing the following pieces: (1) a cover letter explaining what they 
as ratees were being asked to do; (2) a form to collect demographic information about the 
ratee (participant); (3) a form for the ratee to name the five persons who were to complete 
both the LAI and the LEI as observer-raters; (4) five copies of the LAI rating-by-observer 
forms; (5) five copies of the LEI\ (6) five copies of a cover letter; and (7) five envelopes 
(return addressed and postage paid) for completed forms to be sent back directly to the 
researchers. 

Directions to the ratees (the 905 persons who agreed to participate in the study) 
stipulated that the LAI and LEI were to be given to five persons who **(a) Report to you 
either directly or indirectly (or in the event that you do not have five subordinates, they may 
be peers); (b) know you well at work; and (c) who, as far as possible, include females and 
persons from minority groups." 
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These directions call for raters to be persons who know the ratee well at work so 
that both the validity and reliability of ratings would be maximized. Edwards and Sproule 
(1985), for example, found that maximum inteixater agreement among raters occurs as their 
knowledge about the ratee increases. 

For several reasons, the directions allowed ratees to select their own raters. First, 
in contrast to a random selection of raters, it helped assure that the raters were persons who 
knew the ratee fairly well. Second, it guaranteed the credibility of the raters and, therefore, 
the acceptability of their ratings by the ratee and the utility of the. results for professional 
development purposes. Third, empirical research (Wherry & Fryer, 1949; Hollander, 
1956; Waters & Waters, 1970; Edwards, 1990) has shown that friendship does not bias 
evaluations. 

The directions also call for five ratv,rs who are subordinates or, where necessary, 
peers. David Campbell (January 7, 1993), a leadership expert, reported that after "four 
■plus" raters, the average and the standard deviation of raters' scores hardly change. The 
use of subordinates as raters whenever possible is consistent with NCRVE's 
conceptualization of leaders as individuals who, through such noncoercive means as 
persuasion and example, influence the behavior of group members. That is, leaders are 
individuals who have earned followers. Who knows more about an individual's leadership 
behavior and influence than those subordinates who are most impacted? A study by 
Edwards (1992) compared the ratings of subordinates with those of peers on 35 kinds of 
leadership behavior of over 5,000 managers. He found that subordinates' ratings were 
more consistent than those of peers and somewhat more rigorous than peers on many of the 
leadership behaviors. However, subordinates and peers agreed far more than they 
disagreed about the strongest and weakest leadership behaviors of the managers. 

One follow-up was conducted with individuals who had agreed over the telephone 
to participate in the study, but who either had not returned completed forms containing 
demographic information or who had fewer than three observer-raters return completed 
LAI and LEI instruments. (A minimum of three raters was considered essential for reliable 
ratings.) Most of the follow-ups were conducted by telephone; the remainder were sent 
letter reminders. 
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All of the completed instruments were electronically screened for eligibility and then 
scored. To be eligible, the respondent had to report that she or he knew the participant/ 
ratee "very well" or "fairly well" (not "casually" or "not at all") and was a subordinate or 
peer (not a superior) of the ratee. The responses of ineligible raters were eliminated, and if 
this reduced the number of eligible raters below three, the ratee was dropped from the 
sample. 

Table 4-2 summarizes the numbers of participants by each of the three samples at 
key stages of the data collection process. The required number of completed LEI and LAI 
forms (at least three) was received from 77% of the individuals who had agreed over the 
telephone to participate in the study (the ratees). After screening the completed LAI forms 
for eligibiUty, 61% of those who had agreed to participate remained in the three samples. 



Table 4-2 

Numbers of Participants by Sample Groups at 
Key Stages of the Data Collection Process 



Data Collection Stage 


Chief 
Vocational 
Administrators 


Vocational 
Department 
Heads 


Vocational 
Teacher 
Leaders 


Total 


1. Agreed To Participate in 
the Study 


311 


289 


305 


905 


2. Returned a Sufficient 
Number of Responses 


260 


221 


212 


693 


3. Returned a Sufficient 
Number of Eligible 
Responses 


220 


168 


163 


551 


4. Eligible Responses as a 
Percent of those 
Agreeing To Participate 


71 


58 


53 


61 



Establishing Norm Groups 

Combining Samples 

With ratings on the LEI collected from three to five eligible observers for each of 
551 ratees in the three sample groups, the first stage of the data analysis was to determine 
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whether two or more of the three groups could be combined to form norm groups. More 
precisely, were the observer-ratings for the three samples sufficiently different to warrant 
establishing three separate norm groups? 

Ratings on the first six items of the L£'/were averaged for each rater, and then that 
mean rating was averaged across the three to five raters for each ratee. The resulting mean 
rating was considered to be the best estimate available of the ratee' s leadership performance 
and was used in subsequent steps of the analysis. Table 4-3 presents the means and 
standard deviations of LEI ratings (1-6) on the three sample groups. 



Table 4-3 

Means and Standard Deviations of LEI Ratings on the Three Sample Groups 



Statistic 


Chief Vocational 
Administrators 
(n=220) 


Vocational 
Department Heads 
(n=168) 


Vocational 
Teacher Leaders 
(n=163) 


Mean Rating 


4.71 


4.76 


4.80 


Standard 
Deviation 


.56 


.59 


.55 



A one-way ANOVA was conducted to test for differences among the three group 
means. No significant difference's were found at the .05 level. (The probability level of 
finding a difference this great was p=.255.) Given the results of the ANOVA, it was 
decided to combine the three sample groups into one norm group of "Vocational 
Administrators and Vocational Teacher Leaders" (n=551). Table 4-4 summarizes the 
characteristics of this norm group. 

It should be noted that when the data from the LAI was analyzed, the order of the 
sample group means was the same as reported in Table 4-3 for the IE/— the vocational 
teacher leaders were rated highest and the chief vocational administrators lowest. The 
average rating of the 37 lAI attributes was also not significantly different among the threr: 
groups. However, in the case of the LAI, the ratings on the 37 individual attributes were 
quite consistently higher for the vocational teacher leader group. Consequently, for the 
LAI, two norm groups were established consisting of (1) vocational administrators 



(combining chief vocational administrators and vocational department heads) and (2) 
vocational teacher leaders. 

Gender Differences 

Data from the norm group of vocational administrators and vocational teacher 
leaders were examined for gender differences by applying a **t"-test to the mean ratings of 
females (n=248) and males (n=303). A significant difference was found at the .05 level 
w^h the female mean rating (X=4.82, SD=.57) higher than the male mean rating 
(X=4.69, SD=:.57), The actual probability of finding a difference this great by chance 
was .01. 



Table 4«4 

Vocational Administrator and Vocational Teacher Leader Norm Group 



State 

(Institution Type) 


Caucasian 


Other 


Total 


Female 


Male 


Female 


Male 


Arkansas (Technical Colleges) 


8 


20 


2 


1 


31 


Colorado (Community Colleges) 


7 


7 


1 




15 


Florida (Community Colleges) 


38 


38 


5 


6 


87 


Georgia (Technical Colleges) 


21 


17 




1 


39 


Iowa (Community Colleges) 


9 


18 




1 


28 


Illinois (Community Colleges) 


30 


60 


2 


1 


93 


Maryland (Secondary Centers) 


15 


22 


4 


2 


43 


Ohio (Secondary Centers) 


45 


35 




1 


81 


Oklahoma (Secondary Centers) 


15 


23 


2 


2 


42 


Oregon (Community Colleges) 


9 


13 


1 


1 


24 


Tennessee (Technical Colleges) 


15 


16 


1 


2 


34 


Wisconsin (Technical Colleges) 


16 


14 


2 


2 


34 


Totals 


228 


283 


20 


20 


551 



The fact that females are rated higher than males on the LEI is consistent with the 
data from the LAI. While no significant difference was found between genders in the 
vocational teacher leader group with the LA/, women administrators were found to have 
been rated higher than men administrators. Two possible explanations were advanced for 
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this result that might also apply to the LEI finding. First, since it is typically more difficult 
for women than men to attain administrative positions, those who do are likely to be a more 
select group. Second, the attributes (qualities) of individuals consistent with the desired 
facilitating, empowering role of leaders in vocational education are often thought to be 
among the strengths ourculture develops in females. 

Although it is recognized that gender differenr-'^ 'o exist, it is not very useful to 
create separate male and female norms. There is I .ained by women comparing 

themselves with other women (or men with othe- a. more r^^alistic and useful for 

all individuals to compare themselves witli a group of practicing vocational administrators 
and teacher leaders. Idealistically, of course, all leaders, regardless of gender, should 
aspire to be as high as possible on all attributes. 

Ethnic Group Differences 

Although attempts were made to include every available minority group member in 
the chief vocational administrator, vocational department head, and vocational teacher 
leader samples, a total of only forty minority persons became a part of the norm group. 
Given this small number, and the fact that there were several ethnic groups included among 
the forty persons, no attempt was made to test for norm group differences based upon 
ethnic group membership. 

Feedback Calculations 

One part of the individualized LEI feedback report (see Appendix B) compares the 
ratee's average observer-ratings with the ratings of the norm group in terms of normalized 
T-scores and percentiles. Appendix C contains a table which provides normalized T-score 
equivalents to LEI average-observer ratings (as raw scores). 

The table in Appendix C also shows the standard error of measurement of the. 
ratee's average observer-rating. The standard error is a measure of uncertainty of the 
precision of the mean rating of the three to five observers actually used. More precisely, if 
a large number of sets of three to five observers were used to rate the same person (whom 
they knew well at work), in 68% of the cases their average rating would fall between plus 
and minus one standard error As the table in Appendix C shows, the standard error of the 
average observer-ratings is .4 points. 
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Appendix A 
Leader Effectiveness Index 
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LEADER EFFECTIVENESS INDEX 

Jerome Moss, Jr, 
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Marking Directions 




• Use pencil or black or blue pen. 

• Darken the circle completely. 

• Erase cleanly any marks you wish to change or 
X out mark if in pen. 

• Do not make any stray marks on this form. 


Correct Marks 

o#oo 

Incorrect Marks 



SECTION A 

We are seeking your opinion about how effectively an individual is performing as a leaden You will return this form directly to 
the National Center for Research in Vocational Education so the person you are rating will not be able to identify your 
responses. Therefore, we urge you to reflect carefully about each statement and select the rating that best describes the 
person. 

For each of the statements which follow, fill in the circle that best describes the person you are rating. 



SECTION B 

1 . Inspires a shared vision and establishes standards that help the organization achieve 
its next stage of development. For example, creates a sense of purpose, defines 
reality in the larger context, instills shared values and beliefs 



2. Fosters unity, collaboration, and ownership, and recognizes individual and team 
contributions. For example, creates a climate of community, builds morale, sets a 
positive tone, resolves disagreements 



Exercises power effectively and empowers others to act. For example, facilitates 
change, shares authority, nurtures the skills of group members 



4. Exerts influence outside of the organization in order to set the right context for the 
organization. For example, serves as a symbol for the group, secures resources, 
builds coalitions, acts as an advocate 



5. Establishes an environment conducive to learning. For example, provides intellectual 
stimulation, creates a supportive climate for learners, facilitates the professional 
development of staff 

6. Satisfies the job-related needs of members of the organization as individuals. 
For example, respects, trusts, and has confidence in members; adapts leadership 
style to the situation; creates a satisfying work environment 



® 
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7. Overall, how effective is the leadership performance of the person you are rating? 

Thank you for completing this surveyl 

Please return the completed survey directly to National Center for Research in Vocational Education 

460 VoTech Building, 1954 Buford Avenue 
University of Minnesota 
St. Paul, MN 55108 

^ '989, 1993. University of Minnesota 
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Appendix B 
Sample Individualized Feedback Report 



LEI INDIVIDUALIZED FEEDBACK REPORT 



Introduction 

You recently requested five of your subordinates (or peers) who know yoi- well at 
work to rate your leadership performance on the Leader Effectiveness Index (LEI). The 
purpose of this report is to provide you (the ratee) with feedback based upon the completed 
instruments. 

Two types of individualized feedback are contained in this report. Chart 1 shows 
the average of the ratings assigned to your performance by your observer raters. Charf 2 
compares the average rating assigned to you with a norm (comparison) group consisting of 
551 vocational administrators and vocational teacher leaders. 



Average LEI Observer-Rating 

Chart 1 contains the average of your observer ratings. The average is in raw score 
form as contained in the LEL 1 means "Not Effective"; 2 is "Slightly Effective"; 3 is 
"Somewhat Effective"; 4 is "Effective"; 5 is "Very Effective"; and 6 is "Extremely 
Effective." The rating shown on Chart 1 is the mean score of items 1-6 on the LEI 
averaged over the three to five observer-raters who completed and returned the instrument. 
(A minimum of three observer-raters was required to report average ratings.) 

The standard error of the mean of the three to five individual observer-ratings is 
shown as a line through the average observer-rating. The standard error is a measure of the 
uncertainty of the precision of the mean rating of the three to five individual observers 
actually used. More precisely, if a large number of sets of three to five observers were 
used, in 68% of the cases their average rating would fall between plus and minus one 
standard error. Consequently, instead of thinking about an average observer-rating, it is 
more accurate to think of a range of average observer-ratings— the range shown by the line 
representing the standard error. 
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Comparing Observer-Ratings with a Norm Group 



Chart 2 compares your average observer-rating with a norm group consisting of 
551 vocational administrators and vocational teacher leaders. The persons in the norm 
groups were drawn purposively from the following states: Arkansas, Colorado, Florida, 
Georgia, Illinois, Iowa, Maryland, Ohio, Oklahoma, Oregon, Tennessee, and Wisconsin. 
(A complete description of the group and the norming process is contained in Moss, 
Lambrecht, Finch, & Jensrud [in press].) 

On Chart 2, the raw scores of the average observer-ratings of norm group members 
were converted to normalized T-scores. This makes the distribution of the T-scores bell- 
shaped with a mean of 50 and a standard deviation (average deviation around the mean) of 
10. Approximately 68% of the average observer scores fall between 40 and 60 on the T 
scale. About 98% of the scores fall between 30 and 70 on the T scale. 

The line through your leverage observer-rating shows its standard error. The 
standard error is a measure of uncertainty of the precision of the mean rating of the set of 
three to five observers actually used. More precisely, if a large number of sets of three to 
five observers were used, in 68% of the cases thtir average ratings would fall between plus 
and minus one standard error. Consequently, instead of thinking about an average 
observer-rating, it is more accurate to think of a range of average observer-ratings — the 
range shown by the line representing the standard error. One use of the standard error is to 
see whether the line representing plus or minus one standard error crosses the T-score of 
50 (the mean) of the norm group ratings. If it does, the observer T-score rating may be 
considered average in the norm group; if not, the observer-rating is either above or below 
the mean of the norm group. 

Below the T-score there is a percentile scale. Each T-score (and each range of T- 
scores) has an equivalent percentile value. The percentile value of a T-score indicates the 
proportion of individuals in the norm group who scored at or below that T-score. For 
example, if you have a T-score equivalent to the 90 percentile, then 90% of the individuals 
in the norm group have scores equa^ to or lower than yours. More appropriately, if the 
standard error of your T-score represents a range equivalent to the 85 to the 93 percentiles, 
then it might be assumed that between 85% and 93% of the individuals in the norm group 
have scores equal to or less than yours. 
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Using Feedback Results 

Given the results shown on your individualized feedback report, the next steps are 
to utilize that information to plan how you might improve your performance. The 
following steps may be helpful: 

A. Establish developmental goals 

1. Meet with your observer-raters. Explore their ideas for areas of 
performance that might be improved and how it nught be accomplished. 

2. Meet with your mentor(s) and have the same discussion. 

3. Use the Leader Attributes Inventory (LAI) (Moss, Lambrecht, Finch, & 
Jensrud [in press]). This multirater instrument will provide a diagnostic 
assessment of 37 leader attributes that predispose desirable leader behavior. 
It will help pinpoint the attributes that should comprise your developmental 
goals. 

4. Select the three to five attributes or areas of performance with greatest need 
for improvement in the immediate future. 

B . Formulate a leadership development plan 

1 . Using the attributes or the areas of performance to be improved as goals, 
create tentative "action plans" that stipulate the activities, resources needs, 
completion date, and method(s) of measuring progress for each goal. 

2. Review the action plans with your mentor(s). 

3 . Revise the action plan(s). 

C . Initiate the planned activities 
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Chart 2 
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Appendix C 



Table Converting LEI Raw Scores to Normalized T-Scores 
(with Standard Error of Measurement) 
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Table C-1 

LEI Raw Scores to Normalized T-Score Equivalents 

(n=551) 

Standard Error is ±.4 points (raw score) 



Raw Score* 
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5.9 


81 

O 1 


/I O 


41 


5 8 


/D 


/I 1 

4. 1 


jy 


5.7 


79 
/ ^ 


A n 


JO 


5 6 


Do 


Q Q 


JO 


5.5 


UD 


J.O 


JJ 


5.4 




Q 7 


j4 


5.3 


Dl 


J.O 


j2 


5.2 


JO 


J.J 




5.1 


56 


3.4 


30 
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4.4 


44 


2.7 


22 



61 

53 



